{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "Elp6vJr__ml7"
   },
   "source": [
    "# The importance of constraints\n",
    "\n",
    "Constraints determine which potential adversarial examples are valid inputs to the model. When determining the efficacy of an attack, constraints are everything. After all, an attack that looks very powerful may just be generating nonsense. Or, perhaps more nefariously, an attack may generate a real-looking example that changes the original label of the input. That's why you should always clearly define the *constraints* your adversarial examples must meet. "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "wzvF0Rdi_mmD"
   },
   "source": [
    "[![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/QData/TextAttack/blob/master/docs/2notebook/2_Constraints.ipynb)\n",
    "\n",
    "[![View Source on GitHub](https://img.shields.io/badge/github-view%20source-black.svg)](https://github.com/QData/TextAttack/blob/master/docs/2notebook/2_Constraints.ipynb)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "LbiCxZmP_mmD"
   },
   "source": [
    "Please remember to run  **pip3 install textattack[tensorflow]**  in your notebook enviroment before the following codes:\n",
    "\n",
    "### Classes of constraints\n",
    "\n",
    "TextAttack evaluates constraints using methods from three groups:\n",
    "\n",
    "- **Overlap constraints** determine if a perturbation is valid based on character-level analysis. For example, some attacks are constrained by edit distance: a perturbation is only valid if it perturbs some small number of characters (or fewer).\n",
    "\n",
    "- **Grammaticality constraints** filter inputs based on syntactical information. For example, an attack may require that adversarial perturbations do not introduce grammatical errors.\n",
    "\n",
    "- **Semantic constraints** try to ensure that the perturbation is semantically similar to the original input. For example, we may design a constraint that uses a sentence encoder to encode the original and perturbed inputs, and enforce that the sentence encodings be within some fixed distance of one another. (This is what happens in subclasses of `textattack.constraints.semantics.sentence_encoders`.)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "0rOgjT4D_mmE"
   },
   "source": [
    "### A new constraint\n",
    "\n",
    "To add our own constraint, we need to create a subclass of `textattack.constraints.Constraint`. We can implement one of two functions, either `_check_constraint` or `_check_constraint_many`:\n",
    "\n",
    "- `_check_constraint` determines whether candidate `AttackedText` `transformed_text`, transformed from `current_text`, fulfills a desired constraint. It returns either `True` or `False`.\n",
    "- `_check_constraint_many` determines whether each of a list of candidates `transformed_texts` fulfill the constraint relative to `current_text`. This is here in case your constraint can be vectorized. If not, just implement `_check_constraint`, and `_check_constraint` will be executed for each `(transformed_text, current_text)` pair."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "rmRFJKAJ_mmE"
   },
   "source": [
    "### A custom constraint\n",
    "\n",
    "\n",
    "For fun, we're going to see what happens when we constrain an attack to only allow perturbations that substitute out a named entity for another. In linguistics, a **named entity** is a proper noun, the name of a person, organization, location, product, etc. Named Entity Recognition is a popular NLP task (and one that state-of-the-art models can perform quite well). \n",
    "\n",
    "\n",
    "### NLTK and Named Entity Recognition\n",
    "\n",
    "**NLTK**, the Natural Language Toolkit, is a Python package that helps developers write programs that process natural language. NLTK comes with predefined algorithms for lots of linguistic tasks– including Named Entity Recognition.\n",
    "\n",
    "First, we're going to write a constraint class. In the `_check_constraints` method, we're going to use NLTK to find the named entities in both `current_text` and `transformed_text`. We will only return `True` (that is, our constraint is met) if `transformed_text` has substituted one named entity in `current_text` for another.\n",
    "\n",
    "Let's import NLTK and download the required modules:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "IlpPoT3f_mmF",
    "outputId": "18fa2078-412f-40f2-98d7-9646144e8023"
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[nltk_data] Downloading package punkt to /u/lab/jy2ma/nltk_data...\n",
      "[nltk_data]   Package punkt is already up-to-date!\n",
      "[nltk_data] Downloading package maxent_ne_chunker to\n",
      "[nltk_data]     /u/lab/jy2ma/nltk_data...\n",
      "[nltk_data]   Package maxent_ne_chunker is already up-to-date!\n",
      "[nltk_data] Downloading package words to /u/lab/jy2ma/nltk_data...\n",
      "[nltk_data]   Package words is already up-to-date!\n",
      "[nltk_data] Downloading package averaged_perceptron_tagger to\n",
      "[nltk_data]     /u/lab/jy2ma/nltk_data...\n",
      "[nltk_data]   Package averaged_perceptron_tagger is already up-to-\n",
      "[nltk_data]       date!\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "True"
      ]
     },
     "execution_count": 1,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "import nltk\n",
    "nltk.download('punkt') # The NLTK tokenizer\n",
    "nltk.download('maxent_ne_chunker') # NLTK named-entity chunker\n",
    "nltk.download('words') # NLTK list of words\n",
    "nltk.download('averaged_perceptron_tagger')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "L0o8WeGO_mmG"
   },
   "source": [
    "### NLTK NER Example\n",
    "\n",
    "Here's an example of using NLTK to find the named entities in a sentence:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "R4AqTLtM_mmH",
    "outputId": "b5c47011-9c11-45ca-84ea-bd51229e29e3"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "(S\n",
      "  In/IN\n",
      "  2017/CD\n",
      "  ,/,\n",
      "  star/NN\n",
      "  quarterback/NN\n",
      "  (PERSON Tom/NNP Brady/NNP)\n",
      "  led/VBD\n",
      "  the/DT\n",
      "  (ORGANIZATION Patriots/NNP)\n",
      "  to/TO\n",
      "  the/DT\n",
      "  (ORGANIZATION Super/NNP Bowl/NNP)\n",
      "  ,/,\n",
      "  but/CC\n",
      "  lost/VBD\n",
      "  to/TO\n",
      "  the/DT\n",
      "  (ORGANIZATION Philadelphia/NNP Eagles/NNP)\n",
      "  ./.)\n"
     ]
    }
   ],
   "source": [
    "sentence = ('In 2017, star quarterback Tom Brady led the Patriots to the Super Bowl, '\n",
    "           'but lost to the Philadelphia Eagles.')\n",
    "\n",
    "# 1. Tokenize using the NLTK tokenizer.\n",
    "tokens = nltk.word_tokenize(sentence)\n",
    "\n",
    "# 2. Tag parts of speech using the NLTK part-of-speech tagger.\n",
    "tagged = nltk.pos_tag(tokens)\n",
    "\n",
    "# 3. Extract entities from tagged sentence.\n",
    "entities = nltk.chunk.ne_chunk(tagged)\n",
    "print(entities)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "fIlv8hAk_mmI"
   },
   "source": [
    "It looks like `nltk.chunk.ne_chunk` gives us an `nltk.tree.Tree` object where named entities are also `nltk.tree.Tree` objects within that tree. We can take this a step further and grab the named entities from the tree of entities:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "8Zkrctxt_mmI",
    "outputId": "6614f0af-07ed-408d-e5bd-56ac75d3799b"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[Tree('PERSON', [('Tom', 'NNP'), ('Brady', 'NNP')]), Tree('ORGANIZATION', [('Patriots', 'NNP')]), Tree('ORGANIZATION', [('Super', 'NNP'), ('Bowl', 'NNP')]), Tree('ORGANIZATION', [('Philadelphia', 'NNP'), ('Eagles', 'NNP')])]\n"
     ]
    }
   ],
   "source": [
    "# 4. Filter entities to just named entities.\n",
    "named_entities = [entity for entity in entities if isinstance(entity, nltk.tree.Tree)]\n",
    "print(named_entities)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "0Xw3RR9S_mmJ"
   },
   "source": [
    "### Caching with `@functools.lru_cache`\n",
    "\n",
    "A little-known feature of Python 3 is `functools.lru_cache`, a decorator that allows users to easily cache the results of a function in an LRU cache. We're going to be using the NLTK library quite a bit to tokenize, parse, and detect named entities in sentences. These sentences might repeat themselves. As such, we'll use this decorator to cache named entities so that we don't have to perform this expensive computation multiple times."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "2CDthqqK_mmJ"
   },
   "source": [
    "### Putting it all together: getting a list of Named Entity Labels from a sentence\n",
    "\n",
    "Now that we know how to tokenize, parse, and detect named entities using NLTK, let's put it all together into a single helper function. Later, when we implement our constraint, we can query this function to easily get the entity labels from a sentence. We can even use `@functools.lru_cache` to try and speed this process up."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {
    "id": "pXiewC00_mmJ"
   },
   "outputs": [],
   "source": [
    "import functools\n",
    "\n",
    "@functools.lru_cache(maxsize=2**14)\n",
    "def get_entities(sentence):\n",
    "    tokens = nltk.word_tokenize(sentence)\n",
    "    tagged = nltk.pos_tag(tokens)\n",
    "    # Setting `binary=True` makes NLTK return all of the named\n",
    "    # entities tagged as NNP instead of detailed tags like\n",
    "    #'Organization', 'Geo-Political Entity', etc.\n",
    "    entities = nltk.chunk.ne_chunk(tagged, binary=True)\n",
    "    return entities.leaves()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "WviS-Tzs_mmK"
   },
   "source": [
    "And let's test our function to make sure it works:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "qaTYRvEN_mmK",
    "outputId": "5fc4c3e1-4094-4fb3-911c-44469aa2e4e7"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[('Jack', 'NNP'),\n",
       " ('Black', 'NNP'),\n",
       " ('starred', 'VBD'),\n",
       " ('in', 'IN'),\n",
       " ('the', 'DT'),\n",
       " ('2003', 'CD'),\n",
       " ('film', 'NN'),\n",
       " ('classic', 'JJ'),\n",
       " ('``', '``'),\n",
       " ('School', 'NNP'),\n",
       " ('of', 'IN'),\n",
       " ('Rock', 'NNP'),\n",
       " (\"''\", \"''\"),\n",
       " ('.', '.')]"
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "sentence = 'Jack Black starred in the 2003 film classic \"School of Rock\".'\n",
    "get_entities(sentence)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "U_WnKG0w_mmL"
   },
   "source": [
    "We flattened the tree of entities, so the return format is a list of `(word, entity type)` tuples. For non-entities, the `entity_type` is just the part of speech of the word. `'NNP'` is the indicator of a named entity (a proper noun, according to NLTK). Looks like we identified three named entities here: 'Jack' and 'Black', 'School', and 'Rock'. as a 'GPE'. (Seems that the labeler thinks Rock is the name of a place, a city or something.) Whatever technique NLTK uses for named entity recognition may be a bit rough, but it did a pretty decent job here!"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "Btum0b2a_mmL"
   },
   "source": [
    "### Creating our NamedEntityConstraint\n",
    "\n",
    "Now that we know how to detect named entities using NLTK, let's create our custom constraint."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "QL-zniHM_mmM",
    "outputId": "238394ce-556b-4738-932e-28f66a9b29bd"
   },
   "outputs": [],
   "source": [
    "from textattack.constraints import Constraint\n",
    "\n",
    "class NamedEntityConstraint(Constraint):\n",
    "    \"\"\" A constraint that ensures `transformed_text` only substitutes named entities from `current_text` with other named entities.\n",
    "    \"\"\"\n",
    "    def _check_constraint(self, transformed_text, current_text):\n",
    "        transformed_entities = get_entities(transformed_text.text)\n",
    "        current_entities = get_entities(current_text.text)\n",
    "        # If there aren't named entities, let's return False (the attack\n",
    "        # will eventually fail).\n",
    "        if len(current_entities) == 0:\n",
    "            return False\n",
    "        if len(current_entities) != len(transformed_entities):\n",
    "            # If the two sentences have a different number of entities, then \n",
    "            # they definitely don't have the same labels. In this case, the \n",
    "            # constraint is violated, and we return False.\n",
    "            return False\n",
    "        else:\n",
    "            # Here we compare all of the words, in order, to make sure that they match.\n",
    "            # If we find two words that don't match, this means a word was swapped \n",
    "            # between `current_text` and `transformed_text`. That word must be a named entity to fulfill our\n",
    "            # constraint.\n",
    "            current_word_label = None\n",
    "            transformed_word_label = None\n",
    "            for (word_1, label_1), (word_2, label_2) in zip(current_entities, transformed_entities):\n",
    "                if word_1 != word_2:\n",
    "                    # Finally, make sure that words swapped between `x` and `x_adv` are named entities. If \n",
    "                    # they're not, then we also return False.\n",
    "                    if (label_1 not in ['NNP', 'NE']) or (label_2 not in ['NNP', 'NE']):\n",
    "                        return False            \n",
    "            # If we get here, all of the labels match up. Return True!\n",
    "            return True\n",
    "    "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "3Yd6vhoQ_mmM"
   },
   "source": [
    "### Testing our constraint\n",
    "\n",
    "We need to create an attack and a dataset to test our constraint on. We went over all of this in the transformations tutorial, so let's gloss over this part for now."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 506,
     "referenced_widgets": [
      "028a1bd76349401db042003734393f15",
      "8dbb92bbcc294c86bf1abf9b30aff081",
      "f9d3f0645e49489bb5fcf3df19e9e318",
      "46c1bf59866d4c1986a6b9766d3fbcd0",
      "e3686cb138de47c8a0c59d227c8a5de2",
      "c3e743ba8263495b9a2220a663c0d95b",
      "77f9119176b849a2889d77b511d05bb9",
      "ce1041c6aeb04787813047cdfe96934f",
      "982cd396b06942beab0d1470e762c4f1",
      "79c3feb6af904fc7846f16e663c2e31a",
      "bd5efcacdc4546fc85df21b1984dcf6b",
      "05ef230153c14e3c9b49a870d3a880d0",
      "837c7907808f44218204ab9ebb476a33",
      "540b0059291a430a89c9840b2e356381",
      "67472b6dbdf84d4395c5caabfde153c2",
      "0d08c447aefc43928dcb15cd231b1706",
      "4ddb39f1e09c4ceda0a4133a51129a7f",
      "65ccf289d8a5419cad65ddc84ec6588d",
      "3c8d5712dc2643179db9dacbd00eed9e",
      "058e38c6c54e45e48a355182e8e7b444",
      "b1bb77d22f714c13a3d1d2cbc9f954a3",
      "7734cd62444244eaa1a5c43ec1b2e86b",
      "940351711fe14824a761b0edcdbccd03",
      "09540807fb3146e89d27910dfede0a8b",
      "80ed30d01b374937a9d0ba53212f4b89",
      "7924964ee0434b5b831d28f1db35316a",
      "158e2bd9897847c398d85f2287d082c5",
      "ce5fba0d353a44828acf2744a93e6de8",
      "696fc4ade3574c96bceaed4ad0d1d4d5",
      "38c75b09e6f44786be6fdc3a8261d4bd",
      "e27f38bb194542dea1760af2d0d392b3",
      "212008b8dbdc4bc8aca3771fa8d8a90a",
      "2acd0a726d664c46acc0259b3ecc027b",
      "e3ac96af77584554b00d651fec864ea7",
      "fc9636efedbf4dc7ab5c50e4ff1982df",
      "094eecac60754095ba707de5753b6d9c",
      "2a7adb54b4ee4d76868130f3d649119e",
      "6db83ad1be894d40ab44d354d81b9092",
      "66b4054472f74603b1f074d1627f51db",
      "b9bfdfa978224415bfa17ae4caa26b7e",
      "0fe09624a691460ca62c9db3d93eebca",
      "b7debfe9548f4c4cb3af666ed2b30c9d",
      "35f40fc76cab4a00b7cefedb57885c67",
      "1f209b612dbc4a95aaf245ee7f48b2fb",
      "b138694a69eb4db58e2c5a9dbef4473f",
      "f2cac34a441744e980c19b9053bc1063",
      "52850a6b4ba546638aecb620ccb606d6",
      "51939ef0b5644c71a18b82cc42d1f305",
      "6c1f39b9d4d34f96a09829e278e00363",
      "b66f9b8c2bf3443b87e1228a61db576a",
      "2d1f6279a6f5445987c274c206a5239b",
      "a619acc696f441bfa9d63231bc563140",
      "078dcb25a6934e51aaa42efaa91e3eff",
      "a7026c0f6fc14966a5ac64b37bdfe101",
      "06bd7513b45c498dbb6bcf58c29e288a",
      "83cb5137b0a240b4a275d20be02dc663",
      "439d251385c242958d1d8cb918fa66c7",
      "51092de3cc6e4ffea9e4848f0485b70e",
      "2798d8d07f9843639cf380de2d158b39",
      "248d8f77532d447c95f0d2d18e1570ba",
      "808a33ece63244b59874805c6e0c9bb4",
      "e36b18bc25d34af29d824d84800a9e73",
      "43c79089158544659ca1b1b103414348",
      "3cd65e4bc15f4aecbc41bf47d8fda5df",
      "0e893bb3794c4d86a214ffe36ac4fa87",
      "54f6340933a249f3a7b02a55a12d9ed7",
      "e5be6e4ed2744a4681d823a18cd25989",
      "463f39b753cf47a4a8324080b24427f0",
      "31b8831d0f664b729a4464a5603208dd",
      "df527053f0134221a112429594413e01",
      "cd59dab999ba4cc0a0e3613ef25c94f8",
      "71ca23f8a2234cf3b6c8f1e09c843648",
      "8db9eb4babd04974a4e3f7fa591b6fd0",
      "03d8f142a8284590a328102e359cd526",
      "120619e0dbb343ae9478fa54a33fd253",
      "a5740d7a465143c4bec4950dbd186c26",
      "f732207215614cb5b90581f800074b78",
      "794b4ac9d9694bc38e8b253385359507",
      "1fa893538b07462ca3e211c1dd070cd0",
      "c040d588a8384271a282ecb02d4b2f18"
     ]
    },
    "id": "-h4q81Pf_mmO",
    "outputId": "b18408a5-75eb-4087-97b4-77b6700dbbe2"
   },
   "outputs": [
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "d7c034a845e444ec855fff77f91df91b",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "HBox(children=(FloatProgress(value=0.0, description='Downloading', max=922.0, style=ProgressStyle(description_…"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n"
     ]
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "5c23c890797e45559c2443bc4a41cf19",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "HBox(children=(FloatProgress(value=0.0, description='Downloading', max=46753264.0, style=ProgressStyle(descrip…"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n"
     ]
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "8b387bc881384f58959d620ce2ec35fa",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "HBox(children=(FloatProgress(value=0.0, description='Downloading', max=760289.0, style=ProgressStyle(descripti…"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n"
     ]
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "ccd4193eaac44ec58c09bcd7f5a734a9",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "HBox(children=(FloatProgress(value=0.0, description='Downloading', max=156.0, style=ProgressStyle(description_…"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n"
     ]
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "5ba7e4d23fb24e50821fd00c103e5dc0",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "HBox(children=(FloatProgress(value=0.0, description='Downloading', max=25.0, style=ProgressStyle(description_w…"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "textattack: Unknown if model of class <class 'transformers.models.albert.modeling_albert.AlbertForSequenceClassification'> compatible with goal function <class 'textattack.goal_functions.classification.untargeted_classification.UntargetedClassification'>.\n",
      "Using custom data configuration default\n",
      "Reusing dataset ag_news (/p/qdata/jy2ma/.cache/textattack/datasets/ag_news/default/0.0.0/0eeeaaa5fb6dffd81458e293dfea1adba2881ffcbdc3fb56baeb5a892566c29a)\n",
      "textattack: Loading \u001b[94mdatasets\u001b[0m dataset \u001b[94mag_news\u001b[0m, split \u001b[94mtest\u001b[0m.\n"
     ]
    }
   ],
   "source": [
    "# Import the model\n",
    "import transformers\n",
    "from textattack.models.wrappers import HuggingFaceModelWrapper\n",
    "\n",
    "model = transformers.AutoModelForSequenceClassification.from_pretrained(\"textattack/albert-base-v2-ag-news\")\n",
    "tokenizer = transformers.AutoTokenizer.from_pretrained(\"textattack/albert-base-v2-ag-news\")\n",
    "\n",
    "model_wrapper = HuggingFaceModelWrapper(model, tokenizer)\n",
    "\n",
    "# Create the goal function using the model\n",
    "from textattack.goal_functions import UntargetedClassification\n",
    "goal_function = UntargetedClassification(model_wrapper)\n",
    "\n",
    "# Import the dataset\n",
    "from textattack.datasets import HuggingFaceDataset\n",
    "dataset = HuggingFaceDataset(\"ag_news\", None, \"test\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {
    "id": "p0Xp0ixS_mmP"
   },
   "outputs": [],
   "source": [
    "from textattack.transformations import WordSwapEmbedding\n",
    "from textattack.search_methods import GreedyWordSwapWIR\n",
    "from textattack import Attack\n",
    "from textattack.constraints.pre_transformation import RepeatModification, StopwordModification\n",
    "\n",
    "# We're going to the `WordSwapEmbedding` transformation. Using the default settings, this\n",
    "# will try substituting words with their neighbors in the counter-fitted embedding space. \n",
    "transformation = WordSwapEmbedding(max_candidates=20) \n",
    "\n",
    "# We'll use the greedy search with word importance ranking method again\n",
    "search_method = GreedyWordSwapWIR()\n",
    "\n",
    "# Our constraints will be the same as Tutorial 1, plus the named entity constraint\n",
    "constraints = [RepeatModification(),\n",
    "               StopwordModification(),\n",
    "               NamedEntityConstraint(False)]\n",
    "\n",
    "# Now, let's make the attack using these parameters. \n",
    "attack = Attack(goal_function, constraints, transformation, search_method)\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "s_MKNTEh_mmQ"
   },
   "source": [
    "Now, let's use our attack. We're going to attack samples until we achieve 5 successes. (There's a lot to check here, and since we're using a greedy search over all potential word swap positions, each sample will take a few minutes. This will take a few hours to run on a single core.)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "YuLnuXY4_mmQ",
    "outputId": "dfb65f39-145d-4f85-dcf2-ce469b6071b2"
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "textattack: Logging to CSV at path results.csv\n",
      "\n",
      "\n",
      "  0%|          | 0/5 [00:00<?, ?it/s]\u001b[A\u001b[A"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Attack(\n",
      "  (search_method): GreedyWordSwapWIR(\n",
      "    (wir_method):  unk\n",
      "  )\n",
      "  (goal_function):  UntargetedClassification\n",
      "  (transformation):  WordSwapEmbedding(\n",
      "    (max_candidates):  20\n",
      "    (embedding):  WordEmbedding\n",
      "  )\n",
      "  (constraints): \n",
      "    (0): NamedEntityConstraint(\n",
      "        (compare_against_original):  False\n",
      "      )\n",
      "    (1): RepeatModification\n",
      "    (2): StopwordModification\n",
      "  (is_black_box):  True\n",
      ") \n",
      "\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\n",
      "\n",
      " 20%|██        | 1/5 [00:01<00:06,  1.57s/it]\u001b[A\u001b[A\n",
      "\n",
      "[Succeeded / Failed / Skipped / Total] 1 / 0 / 0 / 1:  20%|██        | 1/5 [00:01<00:06,  1.58s/it]\u001b[A\u001b[A"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "--------------------------------------------- Result 1 ---------------------------------------------\n",
      "\u001b[94mBusiness (75%)\u001b[0m --> \u001b[35mSci/tech (61%)\u001b[0m\n",
      "\n",
      "Fears for T N pension after talks Unions representing workers at \u001b[94mTurner\u001b[0m   Newall say they are 'disappointed' after talks with stricken parent firm Federal \u001b[94mMogul\u001b[0m.\n",
      "\n",
      "Fears for T N pension after talks Unions representing workers at \u001b[35mKnapp\u001b[0m   Newall say they are 'disappointed' after talks with stricken parent firm Federal \u001b[35mTitan\u001b[0m.\n",
      "\n",
      "\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\n",
      "\n",
      "[Succeeded / Failed / Skipped / Total] 1 / 1 / 0 / 2:  20%|██        | 1/5 [00:12<00:48, 12.17s/it]\u001b[A\u001b[A"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "--------------------------------------------- Result 2 ---------------------------------------------\n",
      "\u001b[35mSci/tech (100%)\u001b[0m --> \u001b[91m[FAILED]\u001b[0m\n",
      "\n",
      "The Race is On: Second Private Team Sets Launch Date for Human Spaceflight (SPACE.com) SPACE.com - TORONTO, Canada -- A second\\team of rocketeers competing for the  #36;10 million Ansari X Prize, a contest for\\privately funded suborbital space flight, has officially announced the first\\launch date for its manned rocket.\n",
      "\n",
      "\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\n",
      "\n",
      "[Succeeded / Failed / Skipped / Total] 1 / 2 / 0 / 3:  20%|██        | 1/5 [00:18<01:14, 18.64s/it]\u001b[A\u001b[A"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "--------------------------------------------- Result 3 ---------------------------------------------\n",
      "\u001b[35mSci/tech (100%)\u001b[0m --> \u001b[91m[FAILED]\u001b[0m\n",
      "\n",
      "Ky. Company Wins Grant to Study Peptides (AP) AP - A company founded by a chemistry researcher at the University of Louisville won a grant to develop a method of producing better peptides, which are short chains of amino acids, the building blocks of proteins.\n",
      "\n",
      "\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\n",
      "\n",
      "[Succeeded / Failed / Skipped / Total] 1 / 2 / 0 / 3:  40%|████      | 2/5 [00:21<00:32, 10.80s/it]\u001b[A\u001b[A\n",
      "\n",
      "[Succeeded / Failed / Skipped / Total] 2 / 2 / 0 / 4:  40%|████      | 2/5 [00:21<00:32, 10.81s/it]\u001b[A\u001b[A"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "--------------------------------------------- Result 4 ---------------------------------------------\n",
      "\u001b[35mSci/tech (100%)\u001b[0m --> \u001b[92mSports (69%)\u001b[0m\n",
      "\n",
      "\u001b[35mPrediction\u001b[0m \u001b[35mUnit\u001b[0m Helps \u001b[35mForecast\u001b[0m Wildfires (AP) AP - It's barely dawn when \u001b[35mMike\u001b[0m Fitzpatrick starts his shift with a blur of colorful maps, figures and endless charts, but already he knows what the day will bring. Lightning will strike in places he expects. Winds will pick up, moist places will dry and flames will roar.\n",
      "\n",
      "\u001b[92mForesight\u001b[0m \u001b[92mDriving\u001b[0m Helps \u001b[92mExpectations\u001b[0m Wildfires (AP) AP - It's barely dawn when \u001b[92mMeek\u001b[0m Fitzpatrick starts his shift with a blur of colorful maps, figures and endless charts, but already he knows what the day will bring. Lightning will strike in places he expects. Winds will pick up, moist places will dry and flames will roar.\n",
      "\n",
      "\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\n",
      "\n",
      "[Succeeded / Failed / Skipped / Total] 2 / 3 / 0 / 5:  40%|████      | 2/5 [00:26<00:39, 13.11s/it]\u001b[A\u001b[A"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "--------------------------------------------- Result 5 ---------------------------------------------\n",
      "\u001b[35mSci/tech (100%)\u001b[0m --> \u001b[91m[FAILED]\u001b[0m\n",
      "\n",
      "Calif. Aims to Limit Farm-Related Smog (AP) AP - Southern California's smog-fighting agency went after emissions of the bovine variety Friday, adopting the nation's first rules to reduce air pollution from dairy cow manure.\n",
      "\n",
      "\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\n",
      "\n",
      "[Succeeded / Failed / Skipped / Total] 2 / 4 / 0 / 6:  40%|████      | 2/5 [00:55<01:23, 27.95s/it]\u001b[A\u001b[A"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "--------------------------------------------- Result 6 ---------------------------------------------\n",
      "\u001b[35mSci/tech (100%)\u001b[0m --> \u001b[91m[FAILED]\u001b[0m\n",
      "\n",
      "Open Letter Against British Copyright Indoctrination in Schools The British Department for Education and Skills (DfES) recently launched a \"Music Manifesto\" campaign, with the ostensible intention of educating the next generation of British musicians. Unfortunately, they also teamed up with the music industry (EMI, and various artists) to make this popular. EMI has apparently negotiated their end well, so that children in our schools will now be indoctrinated about the illegality of downloading music.The ignorance and audacity of this got to me a little, so I wrote an open letter to the DfES about it. Unfortunately, it's pedantic, as I suppose you have to be when writing to goverment representatives. But I hope you find it useful, and perhaps feel inspired to do something similar, if or when the same thing has happened in your area.\n",
      "\n",
      "\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\n",
      "\n",
      "[Succeeded / Failed / Skipped / Total] 2 / 5 / 0 / 7:  40%|████      | 2/5 [01:15<01:53, 37.90s/it]\u001b[A\u001b[A"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "--------------------------------------------- Result 7 ---------------------------------------------\n",
      "\u001b[35mSci/tech (100%)\u001b[0m --> \u001b[91m[FAILED]\u001b[0m\n",
      "\n",
      "Loosing the War on Terrorism \\\\\"Sven Jaschan, self-confessed author of the Netsky and Sasser viruses, is\\responsible for 70 percent of virus infections in 2004, according to a six-month\\virus roundup published Wednesday by antivirus company Sophos.\"\\\\\"The 18-year-old Jaschan was taken into custody in Germany in May by police who\\said he had admitted programming both the Netsky and Sasser worms, something\\experts at Microsoft confirmed. (A Microsoft antivirus reward program led to the\\teenager's arrest.) During the five months preceding Jaschan's capture, there\\were at least 25 variants of Netsky and one of the port-scanning network worm\\Sasser.\"\\\\\"Graham Cluley, senior technology consultant at Sophos, said it was staggeri ...\\\\\n",
      "\n",
      "\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\n",
      "\n",
      "[Succeeded / Failed / Skipped / Total] 2 / 6 / 0 / 8:  40%|████      | 2/5 [01:38<02:27, 49.02s/it]\u001b[A\u001b[A"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "--------------------------------------------- Result 8 ---------------------------------------------\n",
      "\u001b[35mSci/tech (100%)\u001b[0m --> \u001b[91m[FAILED]\u001b[0m\n",
      "\n",
      "FOAFKey: FOAF, PGP, Key Distribution, and Bloom Filters \\\\FOAF/LOAF  and bloom filters have a lot of interesting properties for social\\network and whitelist distribution.\\\\I think we can go one level higher though and include GPG/OpenPGP key\\fingerpring distribution in the FOAF file for simple web-of-trust based key\\distribution.\\\\What if we used FOAF and included the PGP key fingerprint(s) for identities?\\This could mean a lot.  You include the PGP key fingerprints within the FOAF\\file of your direct friends and then include a bloom filter of the PGP key\\fingerprints of your entire whitelist (the source FOAF file would of course need\\to be encrypted ).\\\\Your whitelist would be populated from the social network as your client\\discovered new identit ...\\\\\n",
      "\n",
      "\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\n",
      "\n",
      "[Succeeded / Failed / Skipped / Total] 2 / 6 / 0 / 8:  60%|██████    | 3/5 [01:39<01:06, 33.04s/it]\u001b[A\u001b[A\n",
      "\n",
      "[Succeeded / Failed / Skipped / Total] 3 / 6 / 0 / 9:  60%|██████    | 3/5 [01:39<01:06, 33.05s/it]\u001b[A\u001b[A\n",
      "\n",
      "[Succeeded / Failed / Skipped / Total] 3 / 6 / 1 / 10:  60%|██████    | 3/5 [01:39<01:06, 33.06s/it]\u001b[A\u001b[A"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "--------------------------------------------- Result 9 ---------------------------------------------\n",
      "\u001b[35mSci/tech (73%)\u001b[0m --> \u001b[91mWorld (69%)\u001b[0m\n",
      "\n",
      "E-mail scam targets police chief Wiltshire \u001b[35mPolice\u001b[0m warns about \"phishing\" after its fraud squad chief was targeted.\n",
      "\n",
      "E-mail scam targets police chief Wiltshire \u001b[91mConstabulary\u001b[0m warns about \"phishing\" after its fraud squad chief was targeted.\n",
      "\n",
      "\n",
      "--------------------------------------------- Result 10 ---------------------------------------------\n",
      "\u001b[94mBusiness (81%)\u001b[0m --> \u001b[37m[SKIPPED]\u001b[0m\n",
      "\n",
      "Card fraud unit nets 36,000 cards In its first two years, the UK's dedicated card fraud unit, has recovered 36,000 stolen cards and 171 arrests - and estimates it saved 65m.\n",
      "\n",
      "\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\n",
      "\n",
      "[Succeeded / Failed / Skipped / Total] 3 / 7 / 1 / 11:  60%|██████    | 3/5 [01:50<01:13, 36.68s/it]\u001b[A\u001b[A"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "--------------------------------------------- Result 11 ---------------------------------------------\n",
      "\u001b[35mSci/tech (100%)\u001b[0m --> \u001b[91m[FAILED]\u001b[0m\n",
      "\n",
      "Group to Propose New High-Speed Wireless Format  LOS ANGELES (Reuters) - A group of technology companies  including Texas Instruments Inc. &lt;TXN.N&gt;, STMicroelectronics  &lt;STM.PA&gt; and Broadcom Corp. &lt;BRCM.O&gt;, on Thursday said they  will propose a new wireless networking standard up to 10 times  the speed of the current generation.\n",
      "\n",
      "\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\n",
      "\n",
      "[Succeeded / Failed / Skipped / Total] 3 / 7 / 1 / 11:  80%|████████  | 4/5 [01:59<00:29, 29.79s/it]\u001b[A\u001b[A\n",
      "\n",
      "[Succeeded / Failed / Skipped / Total] 4 / 7 / 1 / 12:  80%|████████  | 4/5 [01:59<00:29, 29.80s/it]\u001b[A\u001b[A"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "--------------------------------------------- Result 12 ---------------------------------------------\n",
      "\u001b[35mSci/tech (99%)\u001b[0m --> \u001b[94mBusiness (52%)\u001b[0m\n",
      "\n",
      "Apple \u001b[35mLaunches\u001b[0m \u001b[35mGraphics\u001b[0m Software, \u001b[35mVideo\u001b[0m \u001b[35mBundle\u001b[0m  LOS \u001b[35mANGELES\u001b[0m (\u001b[35mReuters\u001b[0m) - Apple \u001b[35mComputer\u001b[0m Inc.&lt;AAPL.\u001b[35mO\u001b[0m&gt; on  \u001b[35mTuesday\u001b[0m began shipping a new program designed to let users  create real-time motion graphics and unveiled a discount  video-editing software bundle featuring its flagship \u001b[35mFinal\u001b[0m \u001b[35mCut\u001b[0m  \u001b[35mPro\u001b[0m software.\n",
      "\n",
      "Apple \u001b[94mStartup\u001b[0m \u001b[94mCharting\u001b[0m Software, \u001b[94mFilm\u001b[0m \u001b[94mPooling\u001b[0m  LOS \u001b[94mFRESNO\u001b[0m (\u001b[94mMsnbc\u001b[0m) - Apple \u001b[94mTeam\u001b[0m Inc.&lt;AAPL.\u001b[94ms\u001b[0m&gt; on  \u001b[94mFriday\u001b[0m began shipping a new program designed to let users  create real-time motion graphics and unveiled a discount  video-editing software bundle featuring its flagship \u001b[94mConclude\u001b[0m \u001b[94mCuts\u001b[0m  \u001b[94mCareers\u001b[0m software.\n",
      "\n",
      "\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\n",
      "\n",
      "[Succeeded / Failed / Skipped / Total] 4 / 7 / 1 / 12: 100%|██████████| 5/5 [02:04<00:00, 24.83s/it]\u001b[A\u001b[A\n",
      "\n",
      "[Succeeded / Failed / Skipped / Total] 5 / 7 / 1 / 13: 100%|██████████| 5/5 [02:04<00:00, 24.83s/it]\u001b[A\u001b[A"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "--------------------------------------------- Result 13 ---------------------------------------------\n",
      "\u001b[35mSci/tech (98%)\u001b[0m --> \u001b[94mBusiness (72%)\u001b[0m\n",
      "\n",
      "Dutch Retailer Beats \u001b[35mApple\u001b[0m to Local \u001b[35mDownload\u001b[0m Market  AMSTERDAM (Reuters) - Free \u001b[35mRecord\u001b[0m \u001b[35mShop\u001b[0m, a Dutch music  retail chain, beat \u001b[35mApple\u001b[0m \u001b[35mComputer\u001b[0m Inc. to market on Tuesday  with the launch of a new download service in Europe's latest  battleground for digital song services.\n",
      "\n",
      "Dutch Retailer Beats \u001b[94mAbel\u001b[0m to Local \u001b[94mAbsolution\u001b[0m Market  AMSTERDAM (Reuters) - Free \u001b[94mRegistering\u001b[0m \u001b[94mDepot\u001b[0m, a Dutch music  retail chain, beat \u001b[94mCobbler\u001b[0m \u001b[94mTypewriters\u001b[0m Inc. to market on Tuesday  with the launch of a new download service in Europe's latest  battleground for digital song services.\n",
      "\n",
      "\n",
      "\n",
      "+-------------------------------+--------+\n",
      "| Attack Results                |        |\n",
      "+-------------------------------+--------+\n",
      "| Number of successful attacks: | 5      |\n",
      "| Number of failed attacks:     | 7      |\n",
      "| Number of skipped attacks:    | 1      |\n",
      "| Original accuracy:            | 92.31% |\n",
      "| Accuracy under attack:        | 53.85% |\n",
      "| Attack success rate:          | 41.67% |\n",
      "| Average perturbed word %:     | 12.45% |\n",
      "| Average num. words per input: | 59.46  |\n",
      "| Avg num queries:              | 206.83 |\n",
      "+-------------------------------+--------+\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "[<textattack.attack_results.successful_attack_result.SuccessfulAttackResult at 0x7f3bf14ceeb0>,\n",
       " <textattack.attack_results.failed_attack_result.FailedAttackResult at 0x7f3beab4abb0>,\n",
       " <textattack.attack_results.failed_attack_result.FailedAttackResult at 0x7f3d0d2b26a0>,\n",
       " <textattack.attack_results.successful_attack_result.SuccessfulAttackResult at 0x7f3bca8ca370>,\n",
       " <textattack.attack_results.failed_attack_result.FailedAttackResult at 0x7f3bfc3de6d0>,\n",
       " <textattack.attack_results.failed_attack_result.FailedAttackResult at 0x7f3be85a8820>,\n",
       " <textattack.attack_results.failed_attack_result.FailedAttackResult at 0x7f3bef64c040>,\n",
       " <textattack.attack_results.failed_attack_result.FailedAttackResult at 0x7f3d0d2abf10>,\n",
       " <textattack.attack_results.successful_attack_result.SuccessfulAttackResult at 0x7f3bec9ef2b0>,\n",
       " <textattack.attack_results.skipped_attack_result.SkippedAttackResult at 0x7f3bf14ce490>,\n",
       " <textattack.attack_results.failed_attack_result.FailedAttackResult at 0x7f3bf6e73ac0>,\n",
       " <textattack.attack_results.successful_attack_result.SuccessfulAttackResult at 0x7f3bf9a23f10>,\n",
       " <textattack.attack_results.successful_attack_result.SuccessfulAttackResult at 0x7f3be944fe50>]"
      ]
     },
     "execution_count": 17,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from textattack.loggers import CSVLogger # tracks a dataframe for us.\n",
    "from textattack.attack_results import SuccessfulAttackResult\n",
    "from textattack import Attacker, AttackArgs\n",
    "\n",
    "attack_args = AttackArgs(num_successful_examples=5, log_to_csv=\"results.csv\", csv_coloring_style=\"html\")\n",
    "attacker = Attacker(attack, dataset, attack_args)\n",
    "\n",
    "attacker.attack_dataset()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "QSUC1fuG_mmR"
   },
   "source": [
    "Now let's visualize our 5 successes in color:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 30,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 481
    },
    "id": "zm4sWkvu_mmR",
    "outputId": "4b1c83b9-55be-4c95-c672-f7f389f0a87e"
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>original_text</th>\n",
       "      <th>perturbed_text</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>Fears for T N pension after talks Unions representing workers at <font color = blue>Turner</font>   Newall say they are 'disappointed' after talks with stricken parent firm Federal <font color = blue>Mogul</font>.</td>\n",
       "      <td>Fears for T N pension after talks Unions representing workers at <font color = purple>Knapp</font>   Newall say they are 'disappointed' after talks with stricken parent firm Federal <font color = purple>Titan</font>.</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td><font color = purple>Prediction</font> <font color = purple>Unit</font> Helps <font color = purple>Forecast</font> Wildfires (AP) AP - It's barely dawn when <font color = purple>Mike</font> Fitzpatrick starts his shift with a blur of colorful maps, figures and endless charts, but already he knows what the day will bring. Lightning will strike in places he expects. Winds will pick up, moist places will dry and flames will roar.</td>\n",
       "      <td><font color = green>Foresight</font> <font color = green>Driving</font> Helps <font color = green>Expectations</font> Wildfires (AP) AP - It's barely dawn when <font color = green>Meek</font> Fitzpatrick starts his shift with a blur of colorful maps, figures and endless charts, but already he knows what the day will bring. Lightning will strike in places he expects. Winds will pick up, moist places will dry and flames will roar.</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8</th>\n",
       "      <td>E-mail scam targets police chief Wiltshire <font color = purple>Police</font> warns about \"phishing\" after its fraud squad chief was targeted.</td>\n",
       "      <td>E-mail scam targets police chief Wiltshire <font color = red>Constabulary</font> warns about \"phishing\" after its fraud squad chief was targeted.</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>11</th>\n",
       "      <td>Apple <font color = purple>Launches</font> <font color = purple>Graphics</font> Software, <font color = purple>Video</font> <font color = purple>Bundle</font>  LOS <font color = purple>ANGELES</font> (<font color = purple>Reuters</font>) - Apple <font color = purple>Computer</font> Inc.&lt;AAPL.<font color = purple>O</font>&gt; on  <font color = purple>Tuesday</font> began shipping a new program designed to let users  create real-time motion graphics and unveiled a discount  video-editing software bundle featuring its flagship <font color = purple>Final</font> <font color = purple>Cut</font>  <font color = purple>Pro</font> software.</td>\n",
       "      <td>Apple <font color = blue>Startup</font> <font color = blue>Charting</font> Software, <font color = blue>Film</font> <font color = blue>Pooling</font>  LOS <font color = blue>FRESNO</font> (<font color = blue>Msnbc</font>) - Apple <font color = blue>Team</font> Inc.&lt;AAPL.<font color = blue>s</font>&gt; on  <font color = blue>Friday</font> began shipping a new program designed to let users  create real-time motion graphics and unveiled a discount  video-editing software bundle featuring its flagship <font color = blue>Conclude</font> <font color = blue>Cuts</font>  <font color = blue>Careers</font> software.</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>12</th>\n",
       "      <td>Dutch Retailer Beats <font color = purple>Apple</font> to Local <font color = purple>Download</font> Market  AMSTERDAM (Reuters) - Free <font color = purple>Record</font> <font color = purple>Shop</font>, a Dutch music  retail chain, beat <font color = purple>Apple</font> <font color = purple>Computer</font> Inc. to market on Tuesday  with the launch of a new download service in Europe's latest  battleground for digital song services.</td>\n",
       "      <td>Dutch Retailer Beats <font color = blue>Abel</font> to Local <font color = blue>Absolution</font> Market  AMSTERDAM (Reuters) - Free <font color = blue>Registering</font> <font color = blue>Depot</font>, a Dutch music  retail chain, beat <font color = blue>Cobbler</font> <font color = blue>Typewriters</font> Inc. to market on Tuesday  with the launch of a new download service in Europe's latest  battleground for digital song services.</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "import pandas as pd\n",
    "pd.options.display.max_colwidth = 480 # increase column width so we can actually read the examples\n",
    "\n",
    "from IPython.core.display import display, HTML\n",
    "\n",
    "logger = attacker.attack_log_manager.loggers[0]\n",
    "successes = logger.df[logger.df[\"result_type\"] == \"Successful\"]\n",
    "display(HTML(successes[['original_text', 'perturbed_text']].to_html(escape=False)))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "TGzNiULA_mmR"
   },
   "source": [
    "### Conclusion\n",
    "\n",
    "Our constraint seems to have done its job: it filtered out attacks that did not swap out a named entity for another, according to the NLTK named entity detector. However, we can see some problems inherent in the detector: it often thinks the title of the news article or the first word of a given sentence is a named entity, probably due to capitalization. "
   ]
  }
 ],
 "metadata": {
  "accelerator": "GPU",
  "colab": {
   "collapsed_sections": [],
   "name": "2_Constraints.ipynb",
   "provenance": []
  },
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.7"
  },
  "widgets": {
   "application/vnd.jupyter.widget-state+json": {
    "028a1bd76349401db042003734393f15": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "HBoxModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HBoxModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HBoxView",
      "box_style": "",
      "children": [
       "IPY_MODEL_f9d3f0645e49489bb5fcf3df19e9e318",
       "IPY_MODEL_46c1bf59866d4c1986a6b9766d3fbcd0"
      ],
      "layout": "IPY_MODEL_8dbb92bbcc294c86bf1abf9b30aff081"
     }
    },
    "03d8f142a8284590a328102e359cd526": {
     "model_module": "@jupyter-widgets/base",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "058e38c6c54e45e48a355182e8e7b444": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "HTMLModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_09540807fb3146e89d27910dfede0a8b",
      "placeholder": "​",
      "style": "IPY_MODEL_940351711fe14824a761b0edcdbccd03",
      "value": " 760k/760k [00:02&lt;00:00, 344kB/s]"
     }
    },
    "05ef230153c14e3c9b49a870d3a880d0": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "HTMLModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_0d08c447aefc43928dcb15cd231b1706",
      "placeholder": "​",
      "style": "IPY_MODEL_67472b6dbdf84d4395c5caabfde153c2",
      "value": " 46.7M/46.7M [00:02&lt;00:00, 21.0MB/s]"
     }
    },
    "06bd7513b45c498dbb6bcf58c29e288a": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "DescriptionStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "078dcb25a6934e51aaa42efaa91e3eff": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "ProgressStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "ProgressStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "bar_color": null,
      "description_width": "initial"
     }
    },
    "094eecac60754095ba707de5753b6d9c": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "HTMLModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_b9bfdfa978224415bfa17ae4caa26b7e",
      "placeholder": "​",
      "style": "IPY_MODEL_66b4054472f74603b1f074d1627f51db",
      "value": " 25.0/25.0 [00:00&lt;00:00, 237B/s]"
     }
    },
    "09540807fb3146e89d27910dfede0a8b": {
     "model_module": "@jupyter-widgets/base",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "0d08c447aefc43928dcb15cd231b1706": {
     "model_module": "@jupyter-widgets/base",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "0e893bb3794c4d86a214ffe36ac4fa87": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "HBoxModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HBoxModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HBoxView",
      "box_style": "",
      "children": [
       "IPY_MODEL_e5be6e4ed2744a4681d823a18cd25989",
       "IPY_MODEL_463f39b753cf47a4a8324080b24427f0"
      ],
      "layout": "IPY_MODEL_54f6340933a249f3a7b02a55a12d9ed7"
     }
    },
    "0fe09624a691460ca62c9db3d93eebca": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "HBoxModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HBoxModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HBoxView",
      "box_style": "",
      "children": [
       "IPY_MODEL_35f40fc76cab4a00b7cefedb57885c67",
       "IPY_MODEL_1f209b612dbc4a95aaf245ee7f48b2fb"
      ],
      "layout": "IPY_MODEL_b7debfe9548f4c4cb3af666ed2b30c9d"
     }
    },
    "120619e0dbb343ae9478fa54a33fd253": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "FloatProgressModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "FloatProgressModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "ProgressView",
      "bar_style": "info",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_794b4ac9d9694bc38e8b253385359507",
      "max": 1,
      "min": 0,
      "orientation": "horizontal",
      "style": "IPY_MODEL_f732207215614cb5b90581f800074b78",
      "value": 1
     }
    },
    "158e2bd9897847c398d85f2287d082c5": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "FloatProgressModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "FloatProgressModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "ProgressView",
      "bar_style": "success",
      "description": "Downloading: 100%",
      "description_tooltip": null,
      "layout": "IPY_MODEL_38c75b09e6f44786be6fdc3a8261d4bd",
      "max": 156,
      "min": 0,
      "orientation": "horizontal",
      "style": "IPY_MODEL_696fc4ade3574c96bceaed4ad0d1d4d5",
      "value": 156
     }
    },
    "1f209b612dbc4a95aaf245ee7f48b2fb": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "HTMLModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_51939ef0b5644c71a18b82cc42d1f305",
      "placeholder": "​",
      "style": "IPY_MODEL_52850a6b4ba546638aecb620ccb606d6",
      "value": " 5.95k/? [00:00&lt;00:00, 16.8kB/s]"
     }
    },
    "1fa893538b07462ca3e211c1dd070cd0": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "DescriptionStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "212008b8dbdc4bc8aca3771fa8d8a90a": {
     "model_module": "@jupyter-widgets/base",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "248d8f77532d447c95f0d2d18e1570ba": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "HTMLModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_3cd65e4bc15f4aecbc41bf47d8fda5df",
      "placeholder": "​",
      "style": "IPY_MODEL_43c79089158544659ca1b1b103414348",
      "value": " 166M/166M [00:10&lt;00:00, 15.5MB/s]"
     }
    },
    "2798d8d07f9843639cf380de2d158b39": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "FloatProgressModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "FloatProgressModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "ProgressView",
      "bar_style": "success",
      "description": "Downloading: 100%",
      "description_tooltip": null,
      "layout": "IPY_MODEL_e36b18bc25d34af29d824d84800a9e73",
      "max": 166373201,
      "min": 0,
      "orientation": "horizontal",
      "style": "IPY_MODEL_808a33ece63244b59874805c6e0c9bb4",
      "value": 166373201
     }
    },
    "2a7adb54b4ee4d76868130f3d649119e": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "ProgressStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "ProgressStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "bar_color": null,
      "description_width": "initial"
     }
    },
    "2acd0a726d664c46acc0259b3ecc027b": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "HBoxModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HBoxModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HBoxView",
      "box_style": "",
      "children": [
       "IPY_MODEL_fc9636efedbf4dc7ab5c50e4ff1982df",
       "IPY_MODEL_094eecac60754095ba707de5753b6d9c"
      ],
      "layout": "IPY_MODEL_e3ac96af77584554b00d651fec864ea7"
     }
    },
    "2d1f6279a6f5445987c274c206a5239b": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "FloatProgressModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "FloatProgressModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "ProgressView",
      "bar_style": "success",
      "description": "Downloading: ",
      "description_tooltip": null,
      "layout": "IPY_MODEL_a7026c0f6fc14966a5ac64b37bdfe101",
      "max": 1662,
      "min": 0,
      "orientation": "horizontal",
      "style": "IPY_MODEL_078dcb25a6934e51aaa42efaa91e3eff",
      "value": 1662
     }
    },
    "31b8831d0f664b729a4464a5603208dd": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "ProgressStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "ProgressStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "bar_color": null,
      "description_width": "initial"
     }
    },
    "35f40fc76cab4a00b7cefedb57885c67": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "FloatProgressModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "FloatProgressModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "ProgressView",
      "bar_style": "success",
      "description": "Downloading: ",
      "description_tooltip": null,
      "layout": "IPY_MODEL_f2cac34a441744e980c19b9053bc1063",
      "max": 2447,
      "min": 0,
      "orientation": "horizontal",
      "style": "IPY_MODEL_b138694a69eb4db58e2c5a9dbef4473f",
      "value": 2447
     }
    },
    "38c75b09e6f44786be6fdc3a8261d4bd": {
     "model_module": "@jupyter-widgets/base",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "3c8d5712dc2643179db9dacbd00eed9e": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "FloatProgressModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "FloatProgressModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "ProgressView",
      "bar_style": "success",
      "description": "Downloading: 100%",
      "description_tooltip": null,
      "layout": "IPY_MODEL_7734cd62444244eaa1a5c43ec1b2e86b",
      "max": 760289,
      "min": 0,
      "orientation": "horizontal",
      "style": "IPY_MODEL_b1bb77d22f714c13a3d1d2cbc9f954a3",
      "value": 760289
     }
    },
    "3cd65e4bc15f4aecbc41bf47d8fda5df": {
     "model_module": "@jupyter-widgets/base",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "439d251385c242958d1d8cb918fa66c7": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "HBoxModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HBoxModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HBoxView",
      "box_style": "",
      "children": [
       "IPY_MODEL_2798d8d07f9843639cf380de2d158b39",
       "IPY_MODEL_248d8f77532d447c95f0d2d18e1570ba"
      ],
      "layout": "IPY_MODEL_51092de3cc6e4ffea9e4848f0485b70e"
     }
    },
    "43c79089158544659ca1b1b103414348": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "DescriptionStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "463f39b753cf47a4a8324080b24427f0": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "HTMLModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_71ca23f8a2234cf3b6c8f1e09c843648",
      "placeholder": "​",
      "style": "IPY_MODEL_cd59dab999ba4cc0a0e3613ef25c94f8",
      "value": " 560000/0 [00:12&lt;00:00, 55536.98 examples/s]"
     }
    },
    "46c1bf59866d4c1986a6b9766d3fbcd0": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "HTMLModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_ce1041c6aeb04787813047cdfe96934f",
      "placeholder": "​",
      "style": "IPY_MODEL_77f9119176b849a2889d77b511d05bb9",
      "value": " 736/736 [00:06&lt;00:00, 115B/s]"
     }
    },
    "4ddb39f1e09c4ceda0a4133a51129a7f": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "HBoxModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HBoxModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HBoxView",
      "box_style": "",
      "children": [
       "IPY_MODEL_3c8d5712dc2643179db9dacbd00eed9e",
       "IPY_MODEL_058e38c6c54e45e48a355182e8e7b444"
      ],
      "layout": "IPY_MODEL_65ccf289d8a5419cad65ddc84ec6588d"
     }
    },
    "51092de3cc6e4ffea9e4848f0485b70e": {
     "model_module": "@jupyter-widgets/base",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "51939ef0b5644c71a18b82cc42d1f305": {
     "model_module": "@jupyter-widgets/base",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "52850a6b4ba546638aecb620ccb606d6": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "DescriptionStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "540b0059291a430a89c9840b2e356381": {
     "model_module": "@jupyter-widgets/base",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "54f6340933a249f3a7b02a55a12d9ed7": {
     "model_module": "@jupyter-widgets/base",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "65ccf289d8a5419cad65ddc84ec6588d": {
     "model_module": "@jupyter-widgets/base",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "66b4054472f74603b1f074d1627f51db": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "DescriptionStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "67472b6dbdf84d4395c5caabfde153c2": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "DescriptionStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "696fc4ade3574c96bceaed4ad0d1d4d5": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "ProgressStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "ProgressStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "bar_color": null,
      "description_width": "initial"
     }
    },
    "6c1f39b9d4d34f96a09829e278e00363": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "HBoxModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HBoxModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HBoxView",
      "box_style": "",
      "children": [
       "IPY_MODEL_2d1f6279a6f5445987c274c206a5239b",
       "IPY_MODEL_a619acc696f441bfa9d63231bc563140"
      ],
      "layout": "IPY_MODEL_b66f9b8c2bf3443b87e1228a61db576a"
     }
    },
    "6db83ad1be894d40ab44d354d81b9092": {
     "model_module": "@jupyter-widgets/base",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "71ca23f8a2234cf3b6c8f1e09c843648": {
     "model_module": "@jupyter-widgets/base",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "7734cd62444244eaa1a5c43ec1b2e86b": {
     "model_module": "@jupyter-widgets/base",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "77f9119176b849a2889d77b511d05bb9": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "DescriptionStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "7924964ee0434b5b831d28f1db35316a": {
     "model_module": "@jupyter-widgets/base",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "794b4ac9d9694bc38e8b253385359507": {
     "model_module": "@jupyter-widgets/base",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "79c3feb6af904fc7846f16e663c2e31a": {
     "model_module": "@jupyter-widgets/base",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "808a33ece63244b59874805c6e0c9bb4": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "ProgressStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "ProgressStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "bar_color": null,
      "description_width": "initial"
     }
    },
    "80ed30d01b374937a9d0ba53212f4b89": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "HBoxModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HBoxModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HBoxView",
      "box_style": "",
      "children": [
       "IPY_MODEL_158e2bd9897847c398d85f2287d082c5",
       "IPY_MODEL_ce5fba0d353a44828acf2744a93e6de8"
      ],
      "layout": "IPY_MODEL_7924964ee0434b5b831d28f1db35316a"
     }
    },
    "837c7907808f44218204ab9ebb476a33": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "ProgressStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "ProgressStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "bar_color": null,
      "description_width": "initial"
     }
    },
    "83cb5137b0a240b4a275d20be02dc663": {
     "model_module": "@jupyter-widgets/base",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "8db9eb4babd04974a4e3f7fa591b6fd0": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "HBoxModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HBoxModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HBoxView",
      "box_style": "",
      "children": [
       "IPY_MODEL_120619e0dbb343ae9478fa54a33fd253",
       "IPY_MODEL_a5740d7a465143c4bec4950dbd186c26"
      ],
      "layout": "IPY_MODEL_03d8f142a8284590a328102e359cd526"
     }
    },
    "8dbb92bbcc294c86bf1abf9b30aff081": {
     "model_module": "@jupyter-widgets/base",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "940351711fe14824a761b0edcdbccd03": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "DescriptionStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "982cd396b06942beab0d1470e762c4f1": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "HBoxModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HBoxModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HBoxView",
      "box_style": "",
      "children": [
       "IPY_MODEL_bd5efcacdc4546fc85df21b1984dcf6b",
       "IPY_MODEL_05ef230153c14e3c9b49a870d3a880d0"
      ],
      "layout": "IPY_MODEL_79c3feb6af904fc7846f16e663c2e31a"
     }
    },
    "a5740d7a465143c4bec4950dbd186c26": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "HTMLModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_c040d588a8384271a282ecb02d4b2f18",
      "placeholder": "​",
      "style": "IPY_MODEL_1fa893538b07462ca3e211c1dd070cd0",
      "value": " 38000/0 [00:00&lt;00:00, 47160.96 examples/s]"
     }
    },
    "a619acc696f441bfa9d63231bc563140": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "HTMLModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_83cb5137b0a240b4a275d20be02dc663",
      "placeholder": "​",
      "style": "IPY_MODEL_06bd7513b45c498dbb6bcf58c29e288a",
      "value": " 3.37k/? [00:00&lt;00:00, 46.4kB/s]"
     }
    },
    "a7026c0f6fc14966a5ac64b37bdfe101": {
     "model_module": "@jupyter-widgets/base",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "b138694a69eb4db58e2c5a9dbef4473f": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "ProgressStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "ProgressStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "bar_color": null,
      "description_width": "initial"
     }
    },
    "b1bb77d22f714c13a3d1d2cbc9f954a3": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "ProgressStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "ProgressStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "bar_color": null,
      "description_width": "initial"
     }
    },
    "b66f9b8c2bf3443b87e1228a61db576a": {
     "model_module": "@jupyter-widgets/base",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "b7debfe9548f4c4cb3af666ed2b30c9d": {
     "model_module": "@jupyter-widgets/base",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "b9bfdfa978224415bfa17ae4caa26b7e": {
     "model_module": "@jupyter-widgets/base",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "bd5efcacdc4546fc85df21b1984dcf6b": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "FloatProgressModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "FloatProgressModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "ProgressView",
      "bar_style": "success",
      "description": "Downloading: 100%",
      "description_tooltip": null,
      "layout": "IPY_MODEL_540b0059291a430a89c9840b2e356381",
      "max": 46747112,
      "min": 0,
      "orientation": "horizontal",
      "style": "IPY_MODEL_837c7907808f44218204ab9ebb476a33",
      "value": 46747112
     }
    },
    "c040d588a8384271a282ecb02d4b2f18": {
     "model_module": "@jupyter-widgets/base",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "c3e743ba8263495b9a2220a663c0d95b": {
     "model_module": "@jupyter-widgets/base",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "cd59dab999ba4cc0a0e3613ef25c94f8": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "DescriptionStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "ce1041c6aeb04787813047cdfe96934f": {
     "model_module": "@jupyter-widgets/base",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "ce5fba0d353a44828acf2744a93e6de8": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "HTMLModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_212008b8dbdc4bc8aca3771fa8d8a90a",
      "placeholder": "​",
      "style": "IPY_MODEL_e27f38bb194542dea1760af2d0d392b3",
      "value": " 156/156 [00:00&lt;00:00, 230B/s]"
     }
    },
    "df527053f0134221a112429594413e01": {
     "model_module": "@jupyter-widgets/base",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "e27f38bb194542dea1760af2d0d392b3": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "DescriptionStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "e3686cb138de47c8a0c59d227c8a5de2": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "ProgressStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "ProgressStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "bar_color": null,
      "description_width": "initial"
     }
    },
    "e36b18bc25d34af29d824d84800a9e73": {
     "model_module": "@jupyter-widgets/base",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "e3ac96af77584554b00d651fec864ea7": {
     "model_module": "@jupyter-widgets/base",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "e5be6e4ed2744a4681d823a18cd25989": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "FloatProgressModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "FloatProgressModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "ProgressView",
      "bar_style": "info",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_df527053f0134221a112429594413e01",
      "max": 1,
      "min": 0,
      "orientation": "horizontal",
      "style": "IPY_MODEL_31b8831d0f664b729a4464a5603208dd",
      "value": 1
     }
    },
    "f2cac34a441744e980c19b9053bc1063": {
     "model_module": "@jupyter-widgets/base",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "f732207215614cb5b90581f800074b78": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "ProgressStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "ProgressStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "bar_color": null,
      "description_width": "initial"
     }
    },
    "f9d3f0645e49489bb5fcf3df19e9e318": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "FloatProgressModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "FloatProgressModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "ProgressView",
      "bar_style": "success",
      "description": "Downloading: 100%",
      "description_tooltip": null,
      "layout": "IPY_MODEL_c3e743ba8263495b9a2220a663c0d95b",
      "max": 736,
      "min": 0,
      "orientation": "horizontal",
      "style": "IPY_MODEL_e3686cb138de47c8a0c59d227c8a5de2",
      "value": 736
     }
    },
    "fc9636efedbf4dc7ab5c50e4ff1982df": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "FloatProgressModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "FloatProgressModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "ProgressView",
      "bar_style": "success",
      "description": "Downloading: 100%",
      "description_tooltip": null,
      "layout": "IPY_MODEL_6db83ad1be894d40ab44d354d81b9092",
      "max": 25,
      "min": 0,
      "orientation": "horizontal",
      "style": "IPY_MODEL_2a7adb54b4ee4d76868130f3d649119e",
      "value": 25
     }
    }
   }
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}
